The present invention relates to interactive hand-held devices, and more particularly to a method and apparatus for modulating video signals with auxiliary data for reception on and use by receivers such as hand-held devices, and providing promotional opportunities and other benefits to users of these receivers from the reception of the signals.
Users of these hand-held devices and other receivers selectively receive auxiliary data for purposes including enjoyment, promotion, transfer of information, data collection, commercial verification, security, education, and transactions or verifications at points of sale, as well as other commercial, personal, entertainment, or amusement purposes collectively referred to herein as “promotional opportunities”.
U.S. Pat. No. 4,807,031 to Broughton et al. (“Broughton”) entitled “Interactive Video Method and Apparatus” relates generally to in-band video broadcasting of commands and other encoded information to interactive devices. The invention described therein relates generally to interactive educational and entertainment systems, and is described in one embodiment in the context of television program control of toys located where there is a television receiver, as within a residence.
To encode control data capable of providing a benefit to a user, Broughton discloses a novel method of luminance or chrominance modulation of a video signal that creates a composite video signal (i.e., a modulated video signal), wherein the video signal is modulated with control data. The novel modulation method alternately raises and lowers the luminance/chrominance of adjacent horizontal scan lines to create a video subcarrier that contains the control data.
In Broughton, the video signal is not being replaced with other data, nor is the data being added as a separate signal along with the video signal. Rather, the video signal itself is modulated to carry the control data. Therefore, the control data is a part of, or contained within, the video signal and yet is imperceptible to the human eye. The encoding method also includes preview and remove circuitry to ensure suitability or the presence of data encoding and removal of data encoding, respectively.
The control data is transmitted either by television broadcast means, or by pre-recorded video players that are connected to a video display. The control data is then received by the video display where at least one video field of the video display is modulated by control data. The control data is then detected with either opto-electronic or radio frequency (RF) detection means that discriminate between the program material and the control data to detect the control data. The detected control data is further reproduced so that the control data can be used with an interactive device.
The encoding method of Broughton may be used to record a “1” or a “0” on every field of a television video signal. Since under NTSC standard the video signal is broadcast at 30 frames a second, there are 60 fields per second resulting in a data rate of 60 bits per second. However, it is often difficult to achieve 60 bits per second of reliable continuous data using this method as will be explained in greater detail below.
In addition to Broughton's data transmission rate being relatively slow, it may not be reliably used for transmitting long data strings to portable devices and other receivers for processing and reproduction. Accordingly, Broughton in one application is typically used by interactive devices to capture auxiliary data that triggers action on the device (i.e., the device acts in a triggered mode). Furthermore, the auxiliary data signals of Broughton are detected asynchronously, as the decoding process on the interactive devices and other receivers (e.g., decoder boxes) are not synchronized to the video signal received from the display device. Moreover, the beginning of a video field was undeterminable by the interactive device of Broughton.
Broughton preferably operates by superimposing a 7875 Hz subcarrier on the video signal, which is at a rate of half the 15750 Hz horizontal retrace frequency. To modulate the video signal with a subcarrier signal, the intensity of one horizontal line is raised and the intensity of the next line in a field is lowered thereby resulting in the 7875 Hz subcarrier signal.
An example of an implementation of Broughton is as follows: an encoder splits each field of a video signal into 8 equal slices, with each slice occupying approximately 2 milliseconds and encoded with the same data bit. The interactive device records and tracks every eighth bit, and thereafter compares the most recent 8 bits (i.e., 1 byte) it receives to the desired combination of 8 bits stored in a table on the interactive device. If there is a match between the two different sets of 8 bits, the interactive device then proceeds to match a second byte and then a third byte. Once all three bytes are matched, action is triggered on the interactive device, which may include a visual or audio notification of a promotional opportunity. Despite the success of Broughton, there is a need in the art for a new apparatus and method for modulating a video signal with data that is faster and more reliable.
Improvements on the method of modulation described in Broughton are described in U.S. Pat. No. 6,094,228 to Ciardullo et al. and U.S. Pat. No. 6,229,572 to Ciardullo et al. (referred to collectively herein as “Ciardullo”). Both Ciardullo patents describe improved methods of modulation wherein the carrier signals relating to control data (i.e., auxiliary data) are inserted on the visual portion of a video signal by changing the luminance of paired lines in opposite directions. Instead of raising and lowering the intensity on an entire scan line of a video signal as in Broughton, Ciardullo uses pseudo noise sequences to raise and lower the intensity on portions on a series of first lines on every other video scan line in a field of a video signal, where the lines paired to the first lines are modulated with the inverse pseudo noise sequences. Ciardullo thereby allows larger amounts of auxiliary data to be modulated in video signals by use of the pseudo noise sequences. Broughton and Ciardullo, which are owned by the assignee of the present invention, are incorporated by reference herein.
Prior efforts by the assignees of the present patent application include U.S. Utility patent application entitled “Interactive Optical Cards and Other Hand-Held Devices with Increased Connectivity”, U.S. Ser. No. 09/489,373, filed Jan. 21, 2000 of Edward J. Koplar and Daniel A. Ciardullo (“Koplar I”), which is incorporated by reference herein. Koplar I relates to various methods and apparatuses for use with promotion opportunities, such as interactive advertising and gaming. Koplar I describes various methods for receiving and providing data and signals to hand-held devices, as well as apparatuses for use with promotional opportunities and methods of using the same.
Another patent application by the assignees of the present invention is U.S. Utility patent application entitled “Universal Methods and Device for Hand-Held Promotional Opportunities”, Ser. No. 09/829,223, filed 9 Apr. 2001 of Edward J. Koplar, Daniel A. Ciardullo, James G. Withers and Christopher E. Chupp (“Koplar II”), which is incorporated by reference herein. Koplar II describes additional methods for receiving and providing data and signals to hand-held devices, as well as apparatuses for receiving promotional opportunities and methods of using the same.
Yet another patent application by the assignees of the present invention is U.S. patent application entitled “RBDS Method and Device for Processing Promotional Opportunities”, Ser. No. 10/126,770, filed on Apr. 19, 2002, of James G. Withers and Alan G. Maltagliati (referred to hereinafter as “Withers”), which is incorporated by reference herein. Withers I describes further improvements to Koplar I and Koplar II including the transmission of auxiliary data to a hand-held device by use of the RBDS system.
For purposes of the present invention, the term “hand-held device” means an interactive device of portable character, preferably of hand-held type that may be carried in the palm by a user, between fingers of the user, or is otherwise intended to be easily grasped and handled manually by the user. By way of example, hand-held devices includes smart cards, mobile phones, personal digital assistants (PDAs), gaming devices and other hand-held devices capable of receiving and processing auxiliary data.
The present invention need not be implemented by manufacturing a customized hand-held device to incorporate functionality of the present invention. Hand-held devices may have a slot that typically allows the device to receive memory cards and sometimes may allow it to use specially designed interface cards to receive other types of information.
A memory card, which may also be referred to as a flash memory card or a storage card, is a small storage medium that typically uses flash memory to store data such as text, pictures, audio, and video for use by small, portable electronic and computing devices. Memory cards on the market as of the date of the present invention include the SD™ (secure digital) card, the CompactFlash® card, the Memory Stick® card, the MultiMediaCard™ (MMC) and the SmartMedia® card. Memory cards are non-volatile solid-state devices that offer a combination of high storage capacity, fast data transfer, increased flexibility, excellent security and small size. Memory cards are typically accepted via a slot on portable devices.
Although slots are primarily used to receive memory cards that only provide data storage, some of the slots' functionality are not so limited and are capable of interfacing with peripheral devices. An example of the foregoing is the SD slot, which was originally intended to provide portable devices with flash memory. However, the slot was written with an open standard so that computer software operating on a portable device may be written to control peripheral devices connected through use of the SD slot, and such hand-held devices with a slot and open protocol are referred to herein as “slotted hand-held devices”. At the time of this invention, SD slots are available in cameras, cell phones, MP3 players and other portable devices.
An interface device is a card capable of insertion into a card slot with a primary purpose of obtaining data or other input from sources not traditionally available to a slotted hand-held device and is referred to hereinafter as an “interface card”. The biggest advantage of manufacturing and using an interface card (i.e., with a slotted hand-held device as opposed to a customized hand-held device) is that the interface card takes advantage of the resources (e.g., an LCD, Internet access, a keypad, etc.) on existing slotted hand-held devices, thereby reducing the manufacturing cost and increasing the functionality of existing such devices.
An interface protocol such as SDIO (the SD input and output) protocol is a standard implemented on various slotted hand-held devices that allows storage media and peripheral devices to be operated through use of an appropriate slot (e.g., the SD slot). For example, a PDA with a SD slot and enabled with the SDIO protocol and related software that has a SD card outfitted with a camera-like device inserted into its SD slot may use a camera-like device to take photographs that are stored on the PDA or are automatically uploaded to a predetermined Internet location.
Another device that may be configured to be a slotted hand-held device is the Nintendo® Gameboy® game unit. As of late 2002, there are approximately 25,000 Nintendo® Game Boy Advance® game units sold every day, and over 100,000,000 Gameboy game units currently in use worldwide. While Gameboy game units are not equipped with a slot such as a SD memory slot, a company operating under the name X-traFun has developed a Bluetooth™-ready Gameboy cartridge outfitted with a SD slot. The cartridge provides a network mechanism to communicate, and the slot enables the reception of storage and interface cards to provide functionality of the present invention.
The term “computer” is also used herein in its broadest possible sense, and may include without limitation a laptop, compact or personal computer, mobile phone, gaming device, personal digital assistant (PDA), or other computer-like device, or other devices using one or more microprocessors or digital processors to achieve a computing or data processing or data manipulative process or comparable or similar functions.